
% United States Bntent and T^^demark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark OCHce 
Addiew: COMMISSIONER OF PATENTS AND TRADEMARKS 
RO. Box 1450 

Alexandria, Viisinia 223 13- 1 4 SO 

WWW.MptO.gov 



APPLICATION NO. 



FILING DATE 



FIRST NAMED INVENTOR 



ATTORNEY DOCKET NO. 



CONFIRMATION NO. 



09/218,554 



12/22/1998 



ERIC COHEN-SOLAL 



PHA-23.595 



2898 



24737 7590 05/22/2003 

PHILIPS ELECTRONICS NORTH AMERICAN CORP 
580 WHITE PLAINS RD 
TARRYTOWN,NY I059I 



EXAMINER 



ROSENDALE, MATTHEW L 



ART UNIT 



PAPER NUMBER 



2612 

DATE MAILED: 05/22/2003 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev.07-0t) 



Office Action Summary 



Application No. 

09/218,554 



Examiner 

Matthew L Rosendale 



Applicant(s) 

COHEN-SOI-AL ET AL 



Art Unit 

2612 



- The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under tlie provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent temi adjustment. See 37 CFR 1 .704(b). 

Status 

1 )□ Responsive to comnnunication(s) filed on . 



2a)n This action is FINAL. 2b)I3 This action is non-final. 

3) 0 Since this application is in condition for allowance except for fornnal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 
Disposition of Claims 

4) S Claim(s) 7-25 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) 0 Claim(s) is/are allowed. 

6) |EI Claim(s) 1-12,14-18 and 20-24 is/are rejected. 

7) ^ Claim(s) 13,19 and 25 is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) n The specification is objected to by the Examiner. 

10)0 The drawing(s) filed on is/are: a)n accepted or b)^ objected to by the Examiner. 

Applicant may not request that any objection to the drawing{s) be held in abeyance. See 37 CFR 1.85(a). 
11 )□ The proposed drawing correction filed on is: a)n approved b)n disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) n The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§ 119 and 120 

13) 0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 

a)nAll b)n Some*c)n None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 1 7.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) n Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) n Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 
Attachment(s) 

1 ) S Notice of References Cited (PTO-892) 4) □ Inten/iew Summary {PTO-41 3) Paper No(s). . 

2) n Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) D Notice of Infomnal Patent Application (PTO-152) 

3) [3 Infomnation Disclosure Statement(s) (PTO-1449) Paper No(s) 6. 6) □ Other: 
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DETAILED ACTION 



Dramngs 

The drawings filed 12/22/98 are informal drawings acceptable for examination purposes 
but formal drawings will be required after allowance. 



The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language, 

1. Claims 1 - 3, 14 - 16, and 20 - 22 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Masunaga et al. 

Referring to claim 1, Masunaga discloses a method of calculating presets of camera 
parameters corresponding to participants in a video conferencing system where the method 
comprises a camera having tilt, pan and zoom parameters, defining a space based upon the layout 
of the video conferencing room where the camera is parmed so that all possible participants can 
be viewed by the camera and so that a location of each participant in the room can be determined 
as shown in figures 4a - 4d. Camera presets are then calculated based on the optimal position of 
each participant with respect to the alignment of the center of the participant with respect to the 
center of the camera view (Col. 5, Line 48 - Col. 7, Line 53). 



Claim Rejections - 35 USC § 102 
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2. Referring to claim 2, Masunaga discloses means of tracking each participant by 
associating a label with each participant. As shown in figures 3a and 3b, each participant is 
labeled with a letter A, B, C, or D. 

3. Referring to claim 3, Masunaga discloses that when the number of attendants change 
such as deleting a participant, the presets for each remaining participants are updated (Col. 8, 
Lines 46 - 53). 

4. Referring to claim 14, Masunaga discloses a video conferencing system comprising a 
camera having tilt, pan and zoom parameters, defining a space based upon the layout of the video 
conferencing room where the camera so that a location of each participant in the room can be 
determined by panning the camera though the room as shown in figures 4a - 4d. Camera presets 
are then calculated based on determining the optimal position of each participant with respect to 
the alignment of the center of the participant and their head with the center of the camera view as 
shown in figures 3a and 3b (Col. 5, Line 48 - Col. 7, Line 53). 

5. Referring to claim 15, Masunaga discloses means of tracking each participant by 
associating a label with each participant. As shown in figures 3a and 3b, each participant is 
labeled with a letter A, B, C, or D. 
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6. Referring to claim 16, Masunaga discloses that when the number of attendants change 
such as deleting a participant, the presets for each remaining participants are updated (Col. 8, 
Lines 46 - 53). 

7. Referring to claim 20, Masunaga discloses a video conferencing system comprising a 
camera having tilt, pan and zoom parameters, defining a space based upon the layout of the video 
conferencing room where the camera so that a location of each participant in the room can be 
determined by panning the camera though the room as shown in figures 4a - 4d. Camera presets 
are then calculated based on determining the optimal position of each participant with respect to 
the alignment of the center of the participant with the center of the camera view as shown in 
figures 3a and 3b (Col. 5, Line 48 - Col. 7, Line 53). 

8. Referring to claim 21, Masunaga discloses means of tracking each participant by 
associating a label with each participant. As shown in figures 3a and 3b, each participant is 
labeled with a letter A, B, C, or D. 

9. Referring to claim 22, Masunaga discloses that when the number of attendants change 
such as deleting a participant, the presets for each remaining participants are updated (Col. 8, 
Lines 46 - 53). 



Claim Rejections - 35 USC §103 
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The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

10. Claims 4, 17, and 23 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Masunaga in view of Maeng. 

Referring to claims 4, 17, and 23, Masunaga discloses a video conferencing apparatus 
that is able to detect individual participants and store camera presets based on the center of the 
participant in the camera's view as shown in figures 3a and 3b. Masunaga does not disclose a 
means to combine participants into one image and the center of the image is used to determine 
the presets. 

However, Maeng discloses a video conferencing apparatus that can combine multiple 
participants as shown in figure 1 1, into one camera view where the center of the detected 
participants is used to determine the preset value of the camera view. Maeng also discloses that 
combining participants in the camera view is advantageous because, in the event of multiple 
patricians talking, they can both be viewed by zooming the camera out and combining the 
images opposed to having the camera move back and forth to individual presets for each 
participant. Another use proposed by Maeng would be for zooming out to view all participants 
in the conferencing room if no participant is taking (Col. 9, Lines 42 - 67). 

Therefore it would have been obvious to use the teachings of Maeng and provide the 
ability of combining participants into singe camera view's and store the camera preset locations 
so that camera can accurately capture all events in the video conference and have the ability to 
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always be able to view all participants who are talking instead of having to switch back and forth 
in the middle of discussion. 

11. Claims 5 - 12, 18, and 24 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Masunaga in view of applicants' conceded prior art ACPA. 

Referring to claim 5, Masunaga discloses a video conferencing apparatus that detects 
participants in the camera's view. Masunaga does not detect participants by providing a digital 
image composed of a plurality of pixels, producing a binary image from the digital image by 
detecting skin colored pixels, removing pixels corresponding to edges in the luminance 
component of the binary image producing binary image components and mapping the binary 
image components into a graph where they are classified as facial and non-facial candidates. 

However, this method of detecting people in a camera's view is well known as taught by 
Abdel-Mottaleb as shown in figure 10. Therefore it would have been obvious to use the 
detection method of Abdel-Mottaleb with the video conferencing device of Masunaga so that 
participants in the view of the camera may be accurately detected. 

12. Referring to claim 6, Abdel-Mottaleb discloses a detection method further comprising a 
step of applying a heuristic of applying a morphological closing operation on each facial 
candidate to produce a closed facial candidate where high variance pixels are determined in the 
facial candidate along with the ratio between the high variance pixels and the total number of 
pixels in the face candidate. The ratio is then compared to a threshold (Col. 6, Lines 13-19). 
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13. Referring to claim 7, Abdel-Mottaleb discloses a detection method comprising a step of 
removing pixels corresponding to edges in the luminance component by applying a mast to a 
plurality of pixels including an examined pixel, determinate the variance between the examined 
pixel and pixels disposed within the mask and comparing the variance to a variance threshold 
(Col. 3, Line 60 - Col. 4, line 28). 

14. Referring to claim 8, Abdel-Mottaleb further discloses that the step of removing pixels 
corresponding to edges in the luminance component is repeated for decreasing variance 
thresholds until a size of the binary image components is below a component size threshold, and 
after each step of removing, the step of classifying components is performed (Col. 4, lines 14 - 



15. Referring to claim 9, Abdel-Mottaleb discloses that the binary image components are 
connected (Col. 4, Lines 14 - 28). 

16. Referring to claim 10, Abdel-Mottaleb discloses that the step of classifying mapped 
binary image components as facial and non- facial types comprises forming a boimding box 
around a classified component of the components and forming a bounding box around a 
classified component, comparing an area of the bounding box to a boimding box threshold, and 
comparing an aspect ratio of the bounding box to an aspect ratio threshold (Col. 4, Lines 29 - 



28). 



63). 
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1 7. Referring to claim 1 1 , Abdel-Mottaleb discloses that the step of mapping binary image 
components into a graph comprises representing each component as a vertex; coimecting vertices 
with an edge when close in space and similar in color, thereby forming a graph (Col. 5, Lines 13 
-49). 

18. Referring to claim 12, Abdel-Mottaleb discloses that each edge has an associated weight 
and further comprises the steps of extracting the minimum spaiming tree of each graph, 
classifying the corresponding binary image components of each graph as either a face or not a 
face, removing the edge in each graph with the greatest weight thereby forming two smaller 
graphs, and repeating the step of classifying the corresponding binary image components for 
each of the smaller graphs until a bounding box around the smaller graph is smaller than a graph 
threshold (Col. 5, Lines 31 - 60). 

19. Referring to claim 18, Masunaga discloses a video conferencing apparatus that detects 
participants in the camera's view. Masunaga does not detect participants by providing a digital 
image composed of a plurality of pixels, producing a binary image from the digital image by 
detecting skin colored pixels, removing pixels corresponding to edges in the luminance 
component of the binary image producing binary image components and mapping the binary 
image components into a graph where they are classified as facial and non- facial candidates. 
However, this method of detecting people in a camera's view is well known as taught by Abdel- 
Mottaleb as shown in figure 10. 
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Therefore it would have been obvious to use the detection method of Abdel-Mottaleb 
with the video conferencing device of Masunaga so that participants in the view of the camera 
may be accurately detected. 

20. Referring to claim 24, Masunaga discloses a video conferencing apparatus that detects 
participants in the camera's view. Masunaga does not detect participants by providing a digital 
image composed of a plurality of pixels, producing a binary image from the digital image by 
detecting skin colored pixels, removing pixels corresponding to edges in the luminance 
component of the binary image producing binary image components and mapping the binary 
image components into a graph where they are classified as facial and non- facial candidates. 
However, this method of detecting people in a camera's view is well known as taught by Abdel- 
Mottaleb as shown in figure 10. 

Therefore it would have been obvious to use the detection method of Abdel-Mottaleb 
with the video conferencing device of Masunaga so that participants in the view of the camera 
may be accurately detected. 

Claim Objections 

Claims 13, 19, 25 objected to as being dependent upon a rejected base claim, but would 
be allowable if rewritten in independent form including all of the limitations of the base claim 
and any intervening claims. 

Referring to claims 13, 19, and 25, the prior art fails to teach or suggest a second camera 
for updating participant presets of a first camera. 
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Conclusion 



The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

Edanami discloses a video conferencing apparatus that can detect participants in a field of 
view and store camera pan, tilt, and zoom preset parameters for each participant detected in the 
conference room. Participants are detected by detecting their heads in the camera's field of 
view. 

Okada 6,343,141, Qian 6,332,033, Kumagai 5,572,595, Okumura 6,055,323, and 
Krishnamurthy 6,275,614 each disclose a convention method of detecting faces or skin areas in 
images by calculating binary images and performing various analyses. 

Any inquiry conceming this communication or earlier communications from the 
examiner should be directed to Matthew L Rosendale whose telephone nimiber is (703) 305- 
4909. The examiner can normally be reached on Monday - Friday 8: OOam-4: 00pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Wendy Garber can be reached on (703) 305-4929. The fax phone numbers for the 
organization where this appUcation or proceeding is assigned are (703) 872-9314 for regular 
communications and (703) 872-9314 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to customer service whose telephone number is (703) 306-0377. 
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